Information processing device, information processing method, and program

ABSTRACT

The present invention relates to an information processing device, an information processing method, and a program for realizing an intuitive operation to issue an instruction to move an object in a virtual three-dimensional space or the like, to an operator who performs operations while looking at an image of the virtual three-dimensional space. 
     The information processing device in this disclosure includes a control unit that controls a CG image generating unit in response to an input of parameters of three axial directions through a three-dimensional operating unit. The control unit changes the amount of control of the CG image generating unit in accordance with the input parameters, to cause an image of the virtual three-dimensional space to vary between where a virtual camera is associated with the three-dimensional operating unit and where an object is associated with the three-dimensional operating unit. This disclosure can be applied to a video editing device for editing broadcast video images.

TECHNICAL FIELD

This disclosure relates to an information processing device, aninformation processing method, and a program, and more particularly, toan information processing device, an information processing method, anda program for enabling an intuitive operation to display a virtualthree-dimensional space formed with CG (computer graphics) by using ajoystick or the like.

BACKGROUND ART

There have been systems that are capable of moving, turning around, androtating the position of an object, the coordinates of a control point,the position of a virtual camera, and the like in a virtualthree-dimensional space formed with CG, by using an operation inputdevice such as a joystick (see Patent Document 1, for example).

CITATION LIST Patent Document

-   Patent Document 1: Japanese Patent Application Laid-Open No.    2003-265858

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

However, the conventional systems might cause the following problems.For example, when an operator tilts the joystick to the right so as tomove an object to the right in a virtual three-dimensional spacedisplayed on a screen, the object does not necessarily move to the righton the screen, depending on the position of a virtual camera. That is,the direction in which an object moves on the screen varies depending onthe relationship between the virtual camera and the axes of thecoordinates indicating the position of the object. Therefore, it isdifficult for the conventional systems to perform an intuitive operationto move an object in a virtual three-dimensional space or the like.

This disclosure has been made in view of those circumstances, and anobject thereof is to realize an intuitive operation for issuing aninstruction to move an object in a virtual three-dimensional space orthe like, to an operator who performs operations while looking at animage of the virtual three-dimensional space.

Solution to Problems

An information processing device as an aspect of this disclosureincludes: a CG image generating unit that performs real-time renderingof a CG image, based on CG descriptive data that defines contents of avirtual three-dimensional space formed with CG; a three-dimensionaloperating unit for inputting parameters of three axial directionsperpendicular to one another; an associating unit that associates avirtual camera or an object in the virtual three-dimensional space as anobject to be controlled with the three-dimensional operating unit; acoordinate mode selecting unit that selects a target system or a sourcesystem as a coordinate mode indicating the coordinate system of anoperation using the three-dimensional operating unit; and a control unitthat, in response to the input of the parameters of the three axialdirections through the three-dimensional operating unit, controls the CGimage generating unit to cause an image of the virtual three-dimensionalspace to vary between where the virtual camera is associated with thethree-dimensional operating unit and where the object is associated withthe three-dimensional operating unit, the variation being caused bychanging the amount of control of the CG image generating unit inaccordance with the input parameters.

In response to the input of the parameters of the three axial directionsthrough the three-dimensional operating unit, the control unit controlsthe CG image generating unit to cause the image of the virtualthree-dimensional space to vary between where the virtual camera isassociated with the three-dimensional operating unit and where theobject is associated with the three-dimensional operating unit, thevariation being caused by reversing the sign of the amount of control ofthe CG image generating unit in accordance with the input parameters.

The information processing device as an aspect of this disclosure mayfurther include an operation mode selecting unit that selects, as anoperation mode using the three-dimensional operating unit, Locsize,locxyz, Rot, Axisloc, asp, shift asp, or shift rot.

The three-dimensional operating unit can perform different kinds ofoperations between where the virtual camera is associated as an objectto be controlled with the three-dimensional operating unit and where theobject is associated with the three-dimensional operating unit.

The information processing device as an aspect of this disclosure mayfurther include a presenting unit that shows an operator the amount ofcontrol corresponding to the parameters input through thethree-dimensional operating unit.

The associating unit may associate the virtual camera or objects in thevirtual three-dimensional space with the three-dimensional operatingunit.

The three-dimensional operating unit may also serve as a user interfaceof digital special effect equipment.

An information processing method as an aspect of this disclosure is aninformation processing method to be performed by an informationprocessing device that includes: a CG image generating unit thatperforms real-time rendering of a CG image, based on CG descriptive datathat defines contents of a virtual three-dimensional space formed withCG; and a three-dimensional operating unit for inputting parameters ofthree axial directions perpendicular to one another. The informationprocessing method includes: an associating step of associating a virtualcamera or an object in the virtual three-dimensional space as an objectto be controlled with the three-dimensional operating unit; a coordinatemode selecting step of selecting a target system or a source system as acoordinate mode indicating the coordinate system of an operation usingthe three-dimensional operating unit; and a control step of, in responseto the input of the parameters of the three axial directions through thethree-dimensional operating unit, controlling the CG image generatingunit to cause an image of the virtual three-dimensional space to varybetween where the virtual camera is associated with thethree-dimensional operating unit and where the object is associated withthe three-dimensional operating unit, the variation being caused bychanging the amount of control of the CG image generating unit inaccordance with the input parameters.

A program as an aspect of this disclosure causes a computer to functionas: a CG image generating unit that performs real-time rendering of a CGimage, based on CG descriptive data defining contents of a virtualthree-dimensional space formed with CG; a three-dimensional operatingunit for inputting parameters of three axial directions perpendicular toone another; an associating unit that associates a virtual camera or anobject in the virtual three-dimensional space as an object to becontrolled with the three-dimensional operating unit; a coordinate modeselecting unit that selects a target system or a source system as acoordinate mode indicating the coordinate system of an operation usingthe three-dimensional operating unit; and a control unit that, inresponse to the input of the parameters of the three axial directionsthrough the three-dimensional operating unit, controls the CG imagegenerating unit to cause an image of the virtual three-dimensional spaceto vary between where the virtual camera is associated with thethree-dimensional operating unit and where the object is associated withthe three-dimensional operating unit, the variation being caused bychanging the amount of control of the CG image generating unit inaccordance with the input parameters.

In an aspect of this disclosure, a virtual camera or an object in avirtual three-dimensional space is associated as an object to becontrolled with a three-dimensional operating unit, and a target systemor a source system is selected as a coordinate mode indicating thecoordinate system of an operation using the three-dimensional operatingunit. Also, in response to an input of parameters of three axialdirections through the three-dimensional operating unit, a CG imagegenerating unit is controlled. The amount of control of the CG imagegenerating unit in accordance with the input parameters is changed tocause an image of the virtual three-dimensional space to vary betweenwhere the virtual camera is associated with the three-dimensionaloperating unit and where the object is associated with thethree-dimensional operating unit.

Effects of the Invention

According to an aspect of this disclosure, it is possible to realize anintuitive operation for issuing instruction to move an object in avirtual three-dimensional space or the like, to the operator whoperforms operations while looking at an image of the virtualthree-dimensional space. Accordingly, instantaneous, high value-addedvideo images can be generated.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing an exemplary configuration of a videoediting device to which this disclosure is applied.

FIG. 2 is an external view showing an exemplary configuration of anoperation input unit.

FIG. 3 is a diagram showing operations that can be or cannot beperformed with a joystick.

FIG. 4 is a diagram showing parameters that are changed in accordancewith an operation of the joystick in a case where a virtual camera is tobe operated.

FIG. 5 is a diagram showing parameters that are changed in accordancewith an operation of the joystick in a case where an object is to beoperated.

FIG. 6 is a diagram showing movement of an object in accordance with anoperation of the joystick in a case where the object is to be operated.

FIG. 7 is a diagram showing movement of an object in accordance with anoperation of the joystick in a case where the object is to be operated.

FIG. 8 is a diagram showing exemplary screen displays for explaining atwo-dimensional moving operation performed by changing TrnsPstLocX andTrnsPstLocY in matFrWF2D.

FIG. 9 is a diagram showing exemplary screen displays for explaining anenlarging operation performed by changing TrnsPstSize in matFrWF2D.

FIG. 10 is a diagram showing exemplary screen displays for explaining arotating operation performed by changing TrnsPstRotation in matFrWF2D.

FIG. 11 is a diagram showing an exemplary multiscreen display.

FIG. 12 is a diagram showing an exemplary multiscreen display.

FIG. 13 is a diagram showing exemplary multiscreen displays.

FIG. 14 is a diagram showing an exemplary multiscreen display.

FIG. 15 is a block diagram showing an exemplary configuration of acomputer.

MODE FOR CARRYING OUT THE INVENTION

In the following, best modes (hereinafter referred to as embodiments)for carrying out this disclosure will bed described in detail withreference to the accompanying drawings.

1. Embodiment

[Exemplary Configuration of a Video Editing Device]

FIG. 1 shows an exemplary configuration of a video editing device as anembodiment.

This video editing device 10 as so-called digital special effectequipment is operated by an operator such as an editor who edits videoimages of television shows, for example, and is designed to be capableof controlling images of virtual three-dimensional spaces created withCG by using an operation input device such as a joystick or a trackballthat is used in editing video images of television shows.

The video editing device 10 includes a CG image generating unit 11, amatrix switch 12, a user interface 13, and a broadcast video generatingunit 14.

Based on CG descriptive data that is generated in advance, the CG imagegenerating unit 11 performs real-time rendering on images of virtualthree-dimensional spaces created with CG, and outputs the resultantimages to the matrix switch 12, under the control of the broadcast videogenerating unit 14.

In accordance with a selecting operation that is input by an operator,the matrix switch 12 selectively outputs, to the broadcast videogenerating unit 14, some of virtual three-dimensional space images inputfrom the CG image generating unit 11 and input images that are inputfrom a VTR, a video server, or the like (not shown).

The user interface 13 includes an operation input unit 21, acorrespondence table 22, and a display unit 23.

FIG. 3 shows an exemplary configuration of the operation input unit 21.The operation input unit 21 includes a joystick 31 and a Z-ring 32 thatinput parameters of the three-dimensional directions, or the x-, y-, andz-directions, and buttons (coordinate mode select buttons 33, operationmode select buttons 34, and the like) for performing various selectingoperations.

An operator can perform an operation to tilt the joystick 31 right andleft, to issue an instruction for movement, enlargement/reduction,rotation, or the like in the x-direction. Also, an operator can performan operation to tilt the joystick 31 back and forth, to issue aninstruction for movement, enlargement/reduction, rotation, or the likein the y-direction. Further, an operator can perform an operation toturn the joystick 31 or the Z-ring 32 clockwise and counterclockwise, toissue an instruction for movement, enlargement/reduction, rotation, orthe like in the z-direction. Instead of the joystick 31, some otherpointing device such as a trackball may be used.

Also, an operator can operate the coordinate mode select buttons 33provided on the operation input unit 21, to select a target system (ascreen coordinate system) or a source system (a coordinate system of avirtual object (a virtual camera or an object in a virtualthree-dimensional space)) as a coordinate mode for an object to beoperated with the joystick 31.

Further, an operator can operate the operation mode select buttons 34 (aLocsize button, a locxyz button, a Rot button, An Axisloc button, an aspbutton, and a shift button) provided on the operation input unit 21, toselect Locsize, locxyz, Rot, Axisloc, asp, shift asp, or shift rot as anoperation mode. It should be noted that “shift asp” means selecting theshift button and the asp button at the same time. Likewise, “shift rot”means selecting the shift button and the rot button at the same time.

The correspondence table 22 stores respective operator IDs associatedwith each object in a virtual three-dimensional space and a virtualcamera. By associating one or more operator IDs with the joystick 31 byoperating a button or the like on the operation input unit 21, one ormore objects in a virtual three-dimensional space or a virtual cameracan be operated with the joystick 31.

FIG. 3 shows the types of operations that are allowed or prohibitedwhere a virtual camera or an object in a virtual three-dimensional space(hereinafter also referred to simply as an object) is associated withthe joystick 31, or where the joystick 31 is to operate the virtualcamera or the object.

Operations that can be performed with the joystick 31 of the operationinput unit 21 include the following eight types: an enlarging/reducingoperation, a three-dimensional moving operation, a rotating operation, arotational-axis moving operation, an aspect ratio changing operation, adistortion setting operation, a perspective setting operation, and atwo-dimensional moving operation. In a case where a virtual camera is tobe operated, of those eight types, seven types of operations, which arethe three-dimensional moving operation, the rotating operation, therotational-axis moving operation, the aspect ratio changing operation,the distortion setting operation, the perspective setting operation, andthe two-dimensional moving operation, are allowed, and theenlarging/reducing operation is prohibited.

In a case where an object is to be operated, of those eight types, threetypes of operations, which are the enlarging/reducing operation, thethree-dimensional moving operation, and the rotating operation, areallowed, and the rotational-axis moving operation, the aspect ratiochanging operation, the distortion setting operation, the perspectivesetting operation, and the two-dimensional moving operation areprohibited.

Any prohibited operation cannot be selected by an operator, or where aprohibited operation is selected, the CG on display is not changed evenif the joystick 31 is tilted. Accordingly, while the overall operationalfeeling remains the same as the operational feeling of conventionaldigital special effect equipment, such changes as to cause problems inCG configurations are prohibited. Thus, an operator can execute editingoperations, without worrying about incorrect operations.

Referring back to FIG. 1, the display unit 23 presents (displays), to anoperator, the numerical value that represents the amount of movement ofan object or the like in a virtual three-dimensional space. The movementoccurs in accordance with an operation using the joystick 31.

The broadcast video generating unit 14 controls the CG image generatingunit 11 in accordance with an operation using the operation input unit21 of the user interface 13, to perform real-time rendering on an imageof a three-dimensional virtual space created with CG, and output theresultant image to the matrix switch 12. In accordance with an editingoperation that is input through the user interface 13, the broadcastvideo generating unit 14 also processes an image input from the matrixswitch 12, and outputs the resultant image to a later stage.

A microcomputer is incorporated into each of the units from the CG imagegenerating unit 11 to the broadcast video generating unit 14, whichconstitute the video editing device 10, and those units are designed tooperate by exchanging control signals with one another.

[Description of Operations]

At the Time of Setting

First, CG descriptive data (Collada format) that is created in advanceis read into the CG image generating unit 11, and, based on the CGdescriptive data, a default Flavor is created.

Here, the CG descriptive data is created beforehand by using CG creatingsoftware, and is converted into a Collada file of the Collada formatwith greater versatility by the CG creating software. The CG descriptivedata contains not only the Collada file but also texture data (a stillimage and a moving image) to be used, and a shader. Also, a Flavor is afile that stores various settings at the time of rendering acorresponding set of CG descriptive data.

After read into the CG image generating unit 11, the Collada file, thetexture data, and the shader contained in the CG descriptive data areconverted into a Native format and are held in an internal memory. Adefault Flavor is then automatically generated.

In a default Flavor, the first correspondence identification number (1),which is called a Manipulator ID, is assigned to the virtual camera tobe used in CG rendering.

After that, desired Flavors can be created manually (through anoperation that is input by using the operation input unit 21 of the userinterface 13). In the Flavors generated here, Manipulator IDs can beset, and a virtual camera, a virtual light, or (an instance of) anobject can be associated with each of the Manipulator IDs.

To perform (update) those settings, CG image generation is preferablyperformed at the same time. As a node (a virtual camera, a virtuallight, or (an instance of) an object) indicated in the displayed list isselected, the frame surrounding the corresponding portion (which is theselected node) in the CG image being output is highlighted. As a result,the selected node becomes clear, and operability is increased.

At the Time of Operating

When a CG image output is used, an operator selects a Flavor through theuser interface 13, and issues a load instruction. The CG imagegenerating unit 11 reads the corresponding CG descriptive data(converted), and real-time rendering is started.

On the operation input unit 21, buttons each having a Manipulator IDassociated therewith exist for the single joystick 31. When the operatorselects one (or more) of those buttons, the node corresponding to theManipulator ID associated with the selected button in the loaded Flavoris to be controlled through an operation of the joystick 31.

In a case where the Manipulator ID associated with the selected buttonis associated with a virtual camera in the Flavor, for example, theentire CG is moved in the output image in accordance with an operationperformed on the joystick 31. In a case where the Manipulator ID isassociated with something other than a virtual camera, the correspondingnode is moved in the CG space, and rendering is performed on the result.

[Vertex Processing to Display the Same Behavior as the Behavior ofDigital Special Effect Equipment]

First, an operation to be performed in a case where the Manipulator IDcorresponding to a virtual camera is selected and the joystick 31 isoperated is described.

FIG. 4 shows parameters that can be changed through various operationsrelated to a virtual camera.

Each row in the drawing shows a type of control signal to be transmittedwhen the joystick 31 is operated in a combination of a coordinate modeand an operation mode. Each control signal shows the value indicatingthe type thereof, the amount of operation of a three-dimensional vector,such as a value of −10 in the x-direction. As the coordinate mode, a Src(source) system or a Tgt (target) system is selected. As the operationmode, axisloc, asp, Locsize, locxyz, Rot, shift asp, or shift rot isselected.

The respective columns of X, Y, and Z show parameters to which thecontrol signals generated when the joystick 31 is operated are directedas instructions.

For example, in a case where a source system is selected as thecoordinate mode while axisloc is selected as the operation mode, when anoperator tilts the joystick 31, three-dimensional vector parametersTrnsLAxisLoc, TrnsLaxisLoc, and TrnsLaxisLoc*−1 are generated inaccordance with the tilting. In a case where a target system is selectedas the coordinate mode while asp is selected as the operation mode, forexample, when an operator tilts the joystick 31, scalar value parametersTrnsLViewX, TrnsLViewY, and TrnsLPersZ are generated in accordance withthe tilting.

To realize a behavior like that of digital special effect equipment,such as moving a virtual camera with the joystick 31 to change thescreen display, the vertex coordinates that are the coordinate data ofan object coordinate system need to be transformed into projectioncoordinates for drawing on the screen in accordance with the followingequation (1):Projectioncoordinates=matFrWF2D*ProjectionMatrix*matFrWF3D*ViewMatrix*WorldMatrix*vertexcoordinates  (1)

In conventional CG, the following equation (2) is used:Projection coordinates=ProjectionMatrix*ViewMatrix*WorldMatrix*vertexcoordinates  (2)

Here, WorldMatrix is a matrix for transforming vertex coordinates in anobject space into world space coordinates through enlargement,reduction, movement, or rotation. In the world space, the relativepositions of a virtual camera, objects, and the like in the space inwhich drawing is to be performed are determined. WorldMatrix isdetermined by the setting of the CG descriptive data or a modeloperation.

ViewMatrix is a matrix for transforming the world coordinates into acoordinate system (view coordinates) in which the X-, Y-, and Z-axes arethe rightward direction, the upward direction, and the sight-linedirection of the virtual camera, respectively, with the origin being theposition of the virtual camera in the CG description data. In the viewcoordinate system, the positions of objects relative to the virtualcamera to be used for drawing are determined. The virtual cameraparameters related to ViewMatrix are determined by the setting of thevirtual camera in the CG descriptive data.

Here, matFrWF3D is a matrix based on a matrix for transforming vertexcoordinates used in digital special effect equipment into projectioncoordinates. By inserting matFrWF3D between ProjectionMatrix andViewMatrix, the operator of the video editing device 10 can add 3Dtransformations by digital special effect equipment to CG video imagescreated by a CG creator. For example, the effect of a mirror placed infront of a secured camera is achieved. A scenery to which the camera isnot directed (or which the CG creator does not intend to capture) can becaptured through the reflection from the mirror (the effect of digitalspecial effect equipment).

ProjectionMatrix is a matrix for transforming view coordinates intoprojection coordinates. The projection coordinates determine display ofan object on the screen. ProjectionMatrix is determined by a viewingangle, an aspect ratio, a front clipping distance, and a back clippingdistance.

Here, matFrWF2D is a matrix based on a matrix for transforming vertexcoordinates used in digital special effect equipment into projectioncoordinates. By adding matFrWF2D after ProjectionMatrix, the operator ofthe video editing device 10 can add 2D transformations by digitalspecial effect equipment to CG video images created by a CG creator.

In matFrWF3D and matFrWF2D, parameters used in conventional digitalspecial effect equipment, and newly defined parameters are used.

The parameters used in digital special effect equipment include thefollowing three-dimensional vectors and scalar values:

Three-Dimensional Vectors

TrnsLSrcLoc (trans-local source location), TrnsLAxisLoc (trans-localaxis spin), TrnsLSrcRot (trans-local source rotation), TrnsLSrcSpin(trans-local source spin), TrnsLTgtLoc (trans-local target location),TrnsLTgtRot (trans-local target rotation), TrnsLTgtSpin (trans-localtarget spin), TrnsGAxisLoc (trans-global axis location), TrnsGSrcRot(trans-global source rotation), TrnsGSrcSpin (trans-global source spin),TrnsGSrcLoc (trans-global source location), TrnsGTgtLoc (trans-globaltarget location), TrnsGTgtRot (trans-global target rotation),TrnsGTgtSpin (trans-global target spin)

Scalar Values

TrnsPreSize (trans pre size), TrnsSkewX (trans skew X), TrnsSkewY (transskew Y), TrnsPreAspect (trans pre aspect), TrnsPreRateX (trans pre rateX), TrnsPreRateY (trans pre rate Y), TrnsLPstLocX (trans-local postlocation X), TrnsLPstLocY (trans-local post location Y), TrnsLPstSize(trans-local post size), TrnsLViewX (trans-local view X), TrnsLViewY(trans-local view Y), TrnsLPersZ (trans-local perspective Z), TrnsGSize(trans-global size), TrnsGPstLocX (trans-global post location X),TrnsGPstLocY (trans-global post location Y), TrnsGPstSize (trans-globalpost size), TrnsGViewX (trans-global view X), TrnsGViewY (trans-globalview Y), TrnsGPersZ (trans-global perspective Z). The initial value ofeach scalar is 1, which is related to a transformation, or 0, which isrelated to a movement.

The newly defined parameters are zeroAxisLoc (zero axis location),ScreenZeroAxisLoc (screen zero axis location), TrnsPstRotation (transpost rotation), ScreenResolutionW (screen resolution W), andScreenResolutionH (screen resolution H).

zeroAxisLoc: a three-dimensional vector. The central coordinates of abounding box (a rectangular parallelepiped formed with the maximumvalues and the minimum values of the respective coordinate axes on whichall objects and the like in the CG space exist) in the entire loadedscene (a route node). The value is assigned when a virtual camera isdetermined (loaded). Even if the central coordinates vary due to amovement of a moving image or an object, zeroAxis does not vary.

ScreenZeroAxisLoc: a four-dimensional vector. The projection coordinatesof zeroAxis, which can be determined byProjectionMatrix*matFrWF3D*zeroAxisLoc. Used in calculating matFrWF2D.Realizes an enlarging/reducing and rotating operation, with the centerbeing ZeroAxisLoc in the screen.

TrnsPstRotation: a scalar value. Used in calculating matFrWF2D. The setvalue of a rotation angle in an rotating operation, with the centerbeing ZeroAxisLoc in the screen.

ScreenResolutionW: a scalar value. The set value of the horizontalresolution of a video image output.

ScreenResolutionH: a scalar value. The set value of the verticalresolution of a video image output.

The arithmetic expressions of matFrWF3D, matLAxis, and matGAxis, whichare matrixes of outputs, are the following arithmetic expressions (3)through (5):matFrWF3D=Tpv·Tgl·Tga·tgr·Tga(−1)·Tgs·Tll·Tla·Tlr·Tla(−1)·Tpre·Tshpre  (3)matLAxis=Tpv·Tgl·Tga·tgr·Tga(−1)·Tgs·Tll·Tla·Tlr  (4)matGAxis=Tpv·Tgl·Tga·tgr  (5)

In the arithmetic expressions (3) through (5), Tgl means T_(GLoc), Tgameans T_(GAxisLoc), tgr means T_(GTgtSpin)·T_(GRot)·T_(GSreSpin), Tgsmeans T_(GSize), Tll means T_(LLOC), Tla means T_(LAxisLoc), and Tlrmeans T_(LTgtSpin)·T_(LRot)·T_(LSreSpin).

Also, Tpv is expressed as Tpv=Matrix4::translation (Vector3(0,0,−zeroAxisLoc.z))*.

                             [Mathematical  Formula  1] $\begin{pmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\0 & 0 & 1 & 0 \\{{- {TrnsViewX}}*{TrnsPersZ}} & {{- {TrnsViewY}}*{TrnsPersZ}} & {TrnsPersZ} & 1\end{pmatrix}*$

Matrix4::Translation(Vector3(0,0,zeroAxisLoc.z))TrnsViewX=TrnsGViewX+TrnsLViewXTrnsViewY=TrnsGViewY+TrnsLViewYTrnsPersZ={10^−5: whenTrnsGPersZ*TrnsLLPersZ<=0,max(sqrt(TrnsGPersZ*TrnsLPersZ),10^−5):others}//

Here, if 10^−5 is a value close to 0 without limit, the value is notlimited to that.T_(GLoC)=Matrix4::translation((TrnsGSrcLoc*TrnsGSize+TrnsGTgtLoc)*Vector3(1,1,−1))((1,1,−1) being the values for adjusting operability)T_(GAxisLoc)=Matrix4::translation(TrnsGAxisLoc*Vector3(1,1,−1))((1,1,−1)being the values for adjusting operability)T _(GTgtSpin)=RotateMVE(TrnsGTgtSpin)T _(GSreSpin)=RotateMVE(TrnsGSrcSpin)T _(GRot)=RotateMVE(TrnsGSrcRot)*RotateMVE(TrnsGTgtRot)T _(GSize)=Matrix4::scale(Vector3(TrnsGSize,TrnsGSize,1))T_(LLoc)=Matrix4::translation((TrnsLSrcLoc*TrnsPreSize+TrnsLTgtLoc)*Vector3(1,1,−1))((1,1,−1)being the values for adjusting operability)T_(AxisLoc)=Matrix4::translation({TrnsLAxisLocX,TrnsLAxisLocY,−TrnsLAxisLocZ})(negative Z being the value for adjusting operability)T _(LTgtspin)=RotateMVE(TrnsLTgtSpin)T _(LSreSpin)=RotateMVE(TrnsLSrcSpin)T _(LRot)=RotateMVE(TrnsLSrcRot)*RotateMVE(TrnsLTgtRot)

                                                             [Mathematical  Formula  2]$T_{pre} = \begin{pmatrix}{{aspectX}*{TrnsPreRateX}*{TrnsPreSize}} & {{TrnsSkewY}*{TrnsPreRateX}*{TrnsPreSize}*{aspectX}} & 0 & 0 \\{{TrnsSkewX}*{TrnsPreRateY}*{TrnsPreSize}*{aspectY}} & {{TrnsPreRateY}*{TrnsPreSize}*{aspectY}} & 0 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{pmatrix}$aspectX=TrnsPreAspectaspectY=1/TrnsPreAspectT_(LAxisLoc)=Matrix4::translation(zeroAxisLoc+TrnsLAxisLoc*Vector3(1,1,−1))T_(GAxisLoc)=Matrix4::translation(zeroAxisLoc+TrnsGAxisLoc*Vector3(1,1,−1))((1,1,−1)being the values for adjusting operability)

The arithmetic expression of matFrWF2D, which is a matrix of an output,is the following arithmetic expression (6):matFrWF2D=matZeroToCENTER*

$\begin{matrix}{\begin{pmatrix}{TrnsPstSizeCos} & {TrnsPstSizeSinH} & 0 & 0 \\{TrnsPstSizeSinW} & {TrnsPstSizeCos} & 0 & 0 \\0 & 0 & 1 & 0 \\{TrnsPstLocX} & {TrnsPstLocY} & 0 & 1\end{pmatrix}*} & \left\lbrack {{Mathematical}\mspace{14mu}{Formula}\mspace{14mu} 3} \right\rbrack\end{matrix}$

matCenterTOZeroMatrix4matCenterTolero=Matrix4::translation(Vector3(ScreenZeroAxisLoc.x/ScreenZeroAxisLoc.w,ScreenZeroAxisLoc.y/ScreenZeroAxisLoc.w,0))Matrix4matZeroToCenter=Matrix4::inverse(matCenterToZero)Vector4ScreenZeroAxisLoc=ProjectionMatrix*matFrWF3D*Vector4(zeroAxisLoc.xyz,1)TrnsPstLocX=TrnsGPstLocX+TrnsLPstLocXTrnsPstLocY=TrnsGPstLocY+TrnsLPstLocYTrnsPstSize=max(TrnsGPstSize*TrnsLPstSize,10^−8)

Here, if 10^−8 is a value close to 0 without limit, the value is notlimited to that.TrnsPstSizeCos=TrnsPstSize*cos(TrnsPstRotation)TrnsPstSizeSinW=TrnsPstSize*sin(TrnsPstRotation)*(ScreenResolutionH/ScreenResolutionW)TrnsPstSizeSinH=TrnsPstSize*sin(TrnsPstRotation)*(ScreenResolutionW/ScreenResolutionH)

Next, an operation to be performed in a case where the Manipulator IDcorresponding to an object is selected and the joystick 31 of theoperation input unit 21 is operated is described. The results of anoperation such as moving an object or the like are described.

FIG. 5 shows parameters that can be changed through various operationsrelated to an object. Each row in the drawing shows a type of controlsignal to be transmitted when the joystick 31 is operated in acombination of a coordinate mode and an operation mode.

Each control signal shows the value indicating the type thereof, theamount of operation of a three-dimensional vector, such as a value of−10 in the x-direction. As the coordinate mode, a Src (source) system ora Tgt (target) system is selected. As the operation mode, axisloc, asp,Locsize, locxyz, Rot, shift asp, or shift rot is selected.

The respective columns of X, Y, and Z show to which parameters thecontrol signals generated when the joystick 31 is operated are directedas instructions.

For example, in a case where a source system is selected as thecoordinate mode and Locsize is selected as the operation mode, when theoperator tilts the joystick 31 in the X-axis direction, a parametersrcMoveX, which indicates the amount of change in the X-axis directionin accordance with the tilting, is generated. Here, srcMoveX means the Xcomponent of srcMove in a later described equation. The same applies tothe other axes and the other rows in the table.

In FIG. 5, there are no parameters to be associated with the items withblank columns. Therefore, any of those items cannot be selected forpreventing incorrect operations, or nothing will occur even if one ofthose items is selected.

FIG. 6 shows the correspondence between operations for an object andmovements of the object. In this drawing, each row shows an combinationof a coordinate mode and an operation mode, and the respective columnsof X, Y, and Z show the movements of the object caused when the joystick31 is operated, as in FIG. 5.

In FIG. 6, the items with blank columns mean that there is no movementin the object when the joystick 31 is operated for those items. However,so as to fill the blank columns, the correspondence between theoperations for the object and the movements of the object may be changedas shown in FIG. 7.

An operation performed on the joystick 31 is reflected by a CG image, asthe matrix (WorldMatrix) of the respective nodes of the object isupdated with the changed parameters. After the operation is reflected,the values are initialized (each of the initialized values of srcMove,srcRotate, tgtMove, and tgtRotate being 0, and each of the initializedvalues of srcScale and tgtScale being 1).

It should be noted that the center of “Rotate (rotation)” of an objectis secured at the center of the bounding box of the object, for the sakeof convenience.

[Reflecting Process]

In a case where a Src (source) system is selected as the coordinatemode, movement, enlargement/reduction, and rotation in the X-, Y-, andZ-axes in the coordinate system inherent to nodes are performed.Matrix matMove=Matrix4::translation(SrcMove)Matrix matScale=Matrix4::scale(SrcScale)Matrix matRotate=Matrix4::rotation(SrcRotate)WorldMatrix=inverse(toCenter)*matRotate*toCenter*WorldMatrixWorldMatrix=matMove*WorldMatrixWorldMatrix=inverse(toCenter)*matScale*toCenter*WorldMatrix

In a case where a Tgt (target) system is selected as the coordinatemode, movement, enlargement/reduction, and rotation are performed in thethree axes, which are the horizontal, vertical, and sight-linedirections of the display that displays a virtual three-dimensionalspace.Matrix matMove=Matrix4::translation(TgtMove)Matrix matScale=Matrix4::scale(TgtScale)Matrix matRotate=Matrix4::rotation(TgtRotate)matWV=inverse(ViewMatrix)*inverse(matFrWF3D)CenterWV=inverse(matWV)*CentertoCenter=Matrix4::translation(CenterWV)WorldMatrix=matWV*inverse(toCenter)*matRotate*toCenter*inverse(matWV)*WorldMatrixWorldMatrix=matWV*matMove*inverse(matWV)*WorldMatrixWorldMatrix=matWV*fromCenter*matScale*toCenter*inverse(matWV)*WorldMatrix

In an operation performed when a Manipulator ID corresponding to avirtual camera is selected, there are several processes to multiply (1,1, −1) as the values for operability adjustment. However, in anoperation performed when a Manipulator ID corresponding to an object isselected, such an adjustment is not performed.

[Display on the Display Unit 23]

The numerical value that is displayed on the display unit 23 andrepresents the amount of movement of an object or the like in a virtualthree-dimensional space caused through an operation using the joystick31 is calculated based on WorldMatrix.

WorldMatrix=//matrix expression

                         [Mathematical  Formula  4] $\begin{pmatrix}{{Vector}\; 4} & {{Col}\; 0} \\{{Vector}\; 4} & {{Col}\; 1} \\{{Vector}\; 4} & {{Col}\; 2} \\{{Vector}\; 4} & {{Col}\; 3}\end{pmatrix}//{{four}\text{-}{dimensional}\mspace{14mu}{vector} \times 4\mspace{14mu}{expressions}}$

//four-dimensional vector×4 expressionsWorldMatrix′=matFrWF3D*ViewMatrix*WorldMatrix=#matrix expression

                         [Mathematical  Formula  5] $\begin{pmatrix}{{Vector}\; 4} & {{Col}\; 0^{\prime}} \\{{Vector}\; 4} & {{Col}\; 1^{\prime}} \\{{Vector}\; 4} & {{Col}\; 2^{\prime}} \\{{Vector}\; 4} & {{Col}\; 3^{\prime}}\end{pmatrix}//{{four}\text{-}{dimensional}\mspace{14mu}{vector} \times 4\mspace{14mu}{expressions}}$

//four-dimensional vector×4 expressions

SrcScale, TgtScale, SrcMove, and TgtMove are three-dimensional vectors,and SrcRotate and TgtRotate are quaternions.Vector3SrcScale=Vector3(length(Col0.xyz,length(Col1.xyz),length(Col2.xyz)))Vector3 TgtScale=matFrWF3D*ViewMatrix*SrcScaleVector3 SrcMove=Vector3(Col3.x,Col3.y,Col3.z)Vector3 TgtMove=matFrWF3D*ViewMatrix*SrcMove

In a case where length(Col0.xyz)>0&&length(Col1.xyz)>0&&length(Col2.xyz)>0,QuatSrcRotate=normalize(Quat(Vector3(normalize(Col0.xyz),normalize(Col1.xyz),normalize(Col2.xyz))))QuatTgtRotate=normalize(Quat(Vector3(normalize(Col0′.xyz),normalize(Col1′.xyz),normalize(Col2′.xyz))))

In cases other than the above,

Quat SrcRotate={0,0,0,1}

Quat TgtRotate={0,0,0,1}

[Description of Functions and Formats]

The functions and formats used in the above descriptions are nowdescribed.

Vector3 is a three-dimensional vector, and is formed with three kinds ofscalar values. The respective scalar values are referred to as x, y, andz.

Vector4 is a four-dimensional vector, and is formed with four kinds ofscalar values. The respective scalar values are referred to as x, y, z,and w.

Matrix is a 4×4 matrix, and is formed with 16 kinds of scalar values.

Quat is a quaternion, and is formed with four kinds of scalar values.The respective scalar values are referred to as x, y, z, and w.

Vector4 v4

v4.xyz means a transformation into a three-dimensional vector having x,y, and z values of a four-dimensional vector.

v4.x means a transformation into a scalar value having an x value of afour-dimensional vector. The same applies to v4.y and the like. The samealso applies to Vector3.

length(Vector3) is a function to output the length of a vector.

normalize(Vector3) is a function to output a normalized vector.

normalize(Quat) is a function to output a normalized quaternion.

inverse( ) means an inverse matrix.

 Matrix RotateMVE (Vector3 R) is for generating a rotation matrix ofdigital special effect equipment, or for generating a matrix forrotation in the following order: the Y-axis, the X-axis, and the Z-axis.R* = {1, 1, −1} are values for operability adjustment.  Matrix RotateMVE(Vector3 R) {  Return [Mathematical Formula 6] $\begin{pmatrix}{{{\cos\left( {R,x} \right)}*{\cos\left( {R,y} \right)}} + {{\sin\left( {R,z} \right)}*{\sin\left( {R,y} \right)}*{\sin\left( {R,x} \right)}}} & {{\sin\left( {R,z} \right)}*{\cos\left( {R,x} \right)}} & {{{- {\cos\left( {R,z} \right)}}*{\sin\left( {R,y} \right)}} + {{\sin\left( {R,z} \right)}*{\cos\left( {R,y} \right)}*{\sin\left( {R,x} \right)}}} & 0 \\{{{- {\sin\left( {R,z} \right)}}*{\cos\left( {R,y} \right)}} + {{\cos\left( {R,z} \right)}*{\sin\left( {R,y} \right)}*{\sin\left( {R,x} \right)}}} & {{\cos\left( {R,z} \right)}*{\cos\left( {R,x} \right)}} & {{{\sin\left( {R,z} \right)}*{\sin\left( {R,y} \right)}} + {{\cos\left( {R,z} \right)}*{\cos\left( {R,y} \right)}*{\sin\left( {R,x} \right)}}} & 0 \\{{\sin\left( {R,y} \right)}*{\cos\left( {R,x} \right)}} & {- {\sin\left( {R,x} \right)}} & {{\cos\left( {R,y} \right)}*{\cos\left( {R,x} \right)}} & 0 \\0 & 0 & 0 & 1\end{pmatrix};$ }  Matrix Matrix4::scale (Vector3 S) is for generating amatrix for enlargement/reduction.  Matrix Matrix4::scale (Vector3 S) { Return [Mathematical Formula 7] $\begin{pmatrix}{S,x} & 0 & 0 & 0 \\0 & {S,y} & 0 & 0 \\0 & 0 & {S,z} & 0 \\0 & 0 & 0 & 1\end{pmatrix};$ }  Matrix Matrix4::translation (Vector3 T) is forgenerating a matrix for movement.  Matrix Matrix4::translation (Vector3T) {  Return [Mathematical Formula 8] $\begin{pmatrix}0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 \\0 & 0 & 0 & 0 \\{T,x} & {T,y} & {T,z} & 1\end{pmatrix};$ }  Matrix Matrix4::rotation (Vector3 R) is forgenerating a rotation matrix that is a matrix for rotation in thefollowing order: the X-axis, the Y-axis, and the Z-axis.  MatrixMatrix4::rotation (Vector3 R) {  Return [Mathematical Formula 9]${\begin{pmatrix}{\cos\left( {R,z} \right)} & {\sin\left( {R,z} \right)} & 0 & 0 \\{- {\sin\left( {R,z} \right)}} & {\cos\left( {R,z} \right)} & 0 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & 1\end{pmatrix}*\begin{pmatrix}{\cos\left( {R,y} \right)} & 0 & {- {\sin\left( {R,y} \right)}} & 0 \\0 & 1 & 0 & 0 \\{\sin\left( {R,y} \right)} & 0 & {\cos\left( {R,y} \right)} & 0 \\0 & 0 & 0 & 1\end{pmatrix}*\begin{pmatrix}1 & 0 & 0 & 0 \\0 & {\cos\left( {R,x} \right)} & {- {\sin\left( {R,x} \right)}} & 0 \\0 & {\sin\left( {R,x} \right)} & {\cos\left( {R,x} \right)} & 0 \\0 & 0 & 0 & 1\end{pmatrix}};$ }

The differences between a case where a source (Src) system is selectedas the coordinate mode and a case where a target (Tgt) system isselected as the coordinate mode are as follows. A source system is acoordinate system based on an object or the like to be operated, and isa coordinate system that changes the orientation thereof in relation tothe appearance of an output image when the object changes theorientation thereof.

For example, in a case where the traveling direction of a vehicle as avehicle-shaped object is the x-axis, the x-axis is always the travelingdirection of the vehicle even if the vehicle faces any direction in a CGvirtual space. Accordingly, when the joystick 31 is tilted in thex-direction so as to move (Translate) the object, movement in thetraveling direction of the vehicle is achieved.

Meanwhile, a target system is a coordinate system based on theappearance of an output image, and the coordinate axes do not vary nomatter how the object or the like is operated, with the x-axis being theapparent right (for example). When the joystick 31 is tilted in thex-direction so as to move the object by using a target coordinatesystem, the object always moves to the apparent right. If the object hasa shape of a vehicle, the object invariably moves to the apparent right,regardless of directions such as rightward, upward, and downwarddirections with respect to the vehicle.

[Examples of Screen Displays]

Next, the effects of CG control performed by the video editing device 10are described.

FIG. 8 shows exemplary screen displays for explaining the effects of atwo-dimensional moving operation performed on a CG three-dimensionalvirtual space, or changes of TrnsPstLocX and TrnsPstLocY in matFrWF2D.Drawing A shows the state prior to the operation, and drawing B showsthe state after the operation.

As is apparent from a comparison between the drawing A and the drawingB, the position of the vanishing point in the three-dimensional virtualspace is the same in the drawing A and the drawing B, and the drawing Bis a drawing formed by moving the drawing A two-dimensionally upward.Conventionally, it was not possible to perform a two-dimensional movingoperation for such a three-dimensional virtual space.

FIG. 9 shows exemplary screen displays for explaining the effects of anenlarging operation performed on a CG three-dimensional virtual space,or a change of TrnsPstSize in matFrWF2D. Drawing A shows the state priorto the operation, and drawing B shows the state after the operation.

As is apparent from a comparison between the drawing A and the drawingB, the entire image is enlarged without a change of the position of anarbitrary reference point (shown as the cross in the upper left portionin the screen in this case). Conventionally, it was not possible toperform an enlarging operation, with such an arbitrary position being areference point.

FIG. 10 shows exemplary screen displays for explaining the effects of arotating operation performed on a CG three-dimensional virtual space, ora change of TrnsPstRotation in matFrWF2D. Drawing A shows the stateprior to the operation, and drawing B shows the state after theoperation.

As is apparent from a comparison between the drawing A and the drawingB, the entire image is enlarged in an arbitrary direction (clockwise inthis example), with the center being an arbitrary point (the cross inthe upper left portion in the screen in this case). Conventionally, itwas not possible to perform such a rotating operation, with the centerbeing an arbitrary point.

By combining the two-dimensional movement, enlargement, and rotationillustrated in FIGS. 8 through 10, it is possible to cope with amultiscreen that displays different video images that are spatiallycontinuous on different monitors.

FIG. 11 shows a situation where a video image of a three-dimensionalvirtual space is vertically and horizontally divided into four, and isdisplayed on four monitors that are of the same size and are verticallyand horizontally arranged.

FIG. 12 shows a situation where a video image of a three-dimensionalvirtual space is divided into four portions in the vertical direction,and is displayed on four monitors that are of the same size and arearranged in parallel.

FIG. 13 shows exemplary multiscreen displays using more monitors thanthe above. Specifically, drawing A shows a situation where divisionalvideo images are displayed on monitors that are of the same size and arearranged in a 3×3 fashion. Drawing B shows a situation where divisionalvideo images are displayed on monitors that are of the same size and arearranged in a tilted 3×3 fashion. Drawing C shows a situation wheredivisional video images are displayed on monitors that are of differentsizes and are randomly arranged. In a case where monitors with differentaspect ratios are made to coexist, the matrix settings ofProjectionMatrix should be adjusted.

FIG. 14 shows an example of use in a case where two monitors arranged inthe vertical direction are set in such a state as to be movable in thehorizontal direction independently of each other. By changing theparameters of matFrWF2D in accordance with the horizontal movements ofthe monitors, it is possible to display an image on a monitor moving ina three-dimensional virtual space.

[Modifications]

Operations using the joystick 31 of the operation input unit 21 can beextended so as to cope not only with the above described movements androtations but also with CG stereo image processing.

Specifically, an operation to adjust the distance between a virtualcamera for the left eye and a virtual camera for the right eye, anoperation to adjust the angle of convergence, and the like are added. Inthis case, when the joystick 31 is tilted forward, for example, thepoint of regard is moved further away (the angle of convergence isnarrowed). When the joystick 31 is tilted backward, the point of regardis moved closer (the angle of convergence is widened). When the joystick31 is tilted to the left, the distance between the left eye and theright eye is shortened. When the joystick 31 is tilted to the right, thedistance between the left eye and the right eye becomes longer.

The matrix switch 12 may be controlled through the user interface 13 sothat control signals are transmitted from the user interface 13 to thematrix switch 12 in accordance with operations by an operator.

Although the broadcast video generating unit 14 controls the CG imagegenerating unit 11 in this embodiment, the CG image generating unit 11may be controlled directly through the user interface 13.

Also, the CG image generating unit 11 may include an image inputterminal that receives an image signal from outside, and may incorporatethe input image (a video signal) into CG or an output image byperforming texture mapping. Here, as the input image, an output from thematrix switch 12 may be received, or one of the outputs from thebroadcast video generating unit 14 may be received. In a case where animage is incorporated into a plane in CG by texture mapping, and theobject containing the plane is an object corresponding to a ManipulatorID in a Flavor, the input image subjected to the texture mapping alsomoves with the object through an operation of the joystick 31.

Further, the contents of the texture mapping may be controlled throughthe user interface 13. The information indicating which input image isto be used when image inputs are provided for the CG image generatingunit 11, the information indicating to which plane the texture mappingis to be directed, and the like are stored in the Flavor, so that theinformation can be reproduced when the Flavor is loaded.

As a component of the video editing device 10, digital special effectequipment may be further provided, and (the channel of) the digitalspecial effect equipment may be included in the objects to be selectedin accordance with a Manipulator ID or an operator for selecting theManipulator ID. Through such a selecting operation, the joystick 31 canalso serve as a CG operating device or a device for operating thedigital special effect equipment.

Further, as components of the video editing device 10, digital specialeffect equipment or CG image generating units 11 may be provided so thatone of those equipment and units can be selected. In this manner, one ofthose equipment and units can be operated with the joystick 31. Further,two or more of those equipment and units may be selected, so that anoperation of the joystick 31 can control two or more of those equipmentand units. For example, a CG image and an image of digital specialeffect equipment are simultaneously moved, and those images aresuperimposed or combined by the broadcast video generating unit, toobtain one output.

In any of the above described cases, the image signal path can bearbitrarily changed by controlling the matrix switch 12, so as todetermine which output of which device is to be used where.

Further, the user interface 13 may be designed to be able to cope withnot only with a virtual camera and objects in a virtualthree-dimensional space generated by the CG image generating unit 11,but also with digital special effect equipment that enlarges and reducesvideo images.

It should be noted that the above described series of processes can beperformed by hardware or software. In a case where the series ofprocesses are performed by software, a computer into which the programfor realizing the software is incorporated is used, or the program forrealizing the software is installed from a program recording medium intoa general-purpose personal computer that can execute various kinds offunctions by installing various kinds of programs.

It should be noted that the above described series of processes can beperformed by hardware or software. In a case where the series ofprocesses are performed by software, a computer into which the programfor realizing the software is incorporated is used, or the program forrealizing the software is installed from a program recording medium intoa general-purpose computer that can execute various kinds of functionsby installing various kinds of programs.

FIG. 15 is a block diagram showing an example hardware configuration ofa computer that performs the above described series of processes inaccordance with a program.

In this computer 100, a CPU (Central Processing Unit) 101, a ROM (ReadOnly Memory) 102, and a RAM (Random Access Memory) 103 are connected toone another by a bus 104.

An input/output interface 105 is further connected to the bus 104. Thefollowing components are connected to the input/output interface 105: aninput unit 106 formed with a keyboard, a mouse, a microphone, and thelike, an output unit 107 formed with a display, a speaker, and the like,a storage unit 108 formed with a hard disk, a nonvolatile memory, or thelike, a communication unit 109 formed with a network interface or thelike, and a drive 110 for driving a removable medium 111 such as amagnetic disk, an optical disk, a magnetooptical disk, a semiconductormemory, or the like.

In the computer 100 having the above configuration, the CPU 101 loads aprogram stored in the storage unit 108 into the RAM 103 via theinput/output interface 105 and the bus 104, and executes the program, toperform the above described series of processes.

The program to be executed by the computer may be a program for carryingout processes in chronological order in accordance with the sequencedescribed in this specification, or a program for carrying out processesin parallel or whenever necessary such as in response to a call.

Also, the program may be executed by one computer, or may be executed bytwo or more computers in a distributed manner. Further, the program maybe transferred to a remote computer and be executed there.

In this specification, a system means an entire apparatus formed withmore than one device.

It should be noted that embodiments in this disclosure are not limitedto the above described embodiment, and various modifications may be madewithout departing from the scope of the disclosure.

REFERENCE SIGNS LIST

10 Video editing device, 11 CG image generating unit, 12 Matrix switch,13 User interface, 14 Broadcast video generating unit, 21 Operationinput unit, 22 Correspondence table, 23 Display unit, 31 Joystick, 32Z-ring, 33 Coordinate mode select buttons, 34 Operation mode selectbuttons

The invention claimed is:
 1. An information processing devicecomprising: a computer graphics (CG) image generating unit configured toperform real-time rendering of a CG image, based on CG descriptive datadefining contents of a virtual three-dimensional space formed with CG; athree-dimensional operating unit configured to be used for inputtingparameters of three axial directions perpendicular to one another forperforming a set of predetermined operations on a virtual camera and onan object in the virtual three-dimensional space; an associating unitconfigured to provide an ID identifying the virtual camera and toprovide an ID identifying the object in the virtual three-dimensionalspace, the associating unit being responsive to the selection of aprovided ID to determine an object to be controlled such that theselection of the virtual camera ID selects the virtual camera as theobject to be controlled with the three-dimensional operating unit andthe selection of the object ID selects the object in the virtualthree-dimensional space as the object to be controlled with thethree-dimensional operating unit; a coordinate mode selecting unitconfigured to select a target system or a source system as a coordinatemode indicating a coordinate system of an operation using thethree-dimensional operating unit; and a control unit configured to, inresponse to the input of the parameters of the three axial directionsthrough the three-dimensional operating unit, control the CG imagegenerating unit to cause an image of the virtual three-dimensional spaceto vary in response to one set of instructions when the virtual camerais selected as the object to be controlled with the three-dimensionaloperating unit and in response to another set of instructions when theobject in the virtual three-dimensional space is selected as the objectto be controlled with the three-dimensional operating unit, thevariation in the image of the virtual three-dimensional space beingcaused by changing the amount of control of the CG image generating unitin accordance with the input parameters, and wherein the one set ofinstructions allows some of the predetermined operations in the set tobe performed on the virtual camera and prohibits others of thepredetermined operations, and wherein the other set of instructionsallows at least the prohibited operations in the set to be performed onthe object in the virtual three-dimensional space.
 2. The informationprocessing device according to claim 1, wherein, in response to theinput of the parameters of the three axial directions through thethree-dimensional operating unit, the control unit controls the CG imagegenerating unit to cause the image of the virtual three-dimensionalspace to vary by reversing the sign of the amount of control of the CGimage generating unit in accordance with the input parameters.
 3. Theinformation processing device according to claim 1, further comprisingan operation mode selecting unit configured to select, as an operationmode using the three-dimensional operating unit, Locsize, locxyz, Rot,Axisloc, asp, shift asp, or shift rot.
 4. The information processingdevice according to claim 1, further comprising a presenting unitconfigured to show an operator the amount of control corresponding tothe parameters input through the three-dimensional operating unit. 5.The information processing device according to claim 1, wherein thethree-dimensional operating unit serves as a user interface of digitalspecial effect equipment.
 6. An information processing method to beperformed by an information processing device that includes: a computergraphics (CG) image generating unit that performs real-time rendering ofa CG image, based on CG descriptive data defining contents of a virtualthree-dimensional space formed with CG; and a three-dimensionaloperating unit for inputting parameters of three axial directionsperpendicular to one another for performing a set of predeterminedoperations on a virtual camera and on an object in the virtualthree-dimensional space, the information processing method comprising: aproviding an ID identifying the virtual camera and providing an IDidentifying the object in the virtual three-dimensional space;determining an object to be controlled in response to the selection of aprovided ID such that the selection of the virtual camera ID selects thevirtual camera as the object to be controlled with the three-dimensionaloperating unit and the selection of the object ID selects the object inthe virtual three-dimensional space as the object to be controlled withthe three-dimensional operating unit; a coordinate mode selecting stepof selecting a target system or a source system as a coordinate modeindicating a coordinate system of an operation using thethree-dimensional operating unit; and a control step of, in response tothe input of the parameters of the three axial directions through thethree-dimensional operating unit, controlling the CG image generatingunit to cause an image of the virtual three-dimensional space to vary inresponse to one set of instructions when the virtual camera is selectedas the object to be controlled with the three-dimensional operating unitand in response to another set of instructions when the object in thevirtual three-dimensional space is associated as the object to becontrolled with the three-dimensional operating unit, the variation inthe image of the virtual three-dimensional space being caused bychanging the amount of control of the CG image generating unit inaccordance with the input parameters, and wherein the one set ofinstructions allows some of the predetermined operations in the set tobe performed on the virtual camera and prohibits others of thepredetermined operations, and wherein the other set of instructionsallows at least the prohibited operations in the set to be performed onthe object in the virtual three-dimensional space.
 7. A non-transitorycomputer-readable medium on which is stored a program for causing acomputer to function as: a computer graphics (CG) image generating unitconfigured to perform real-time rendering of a CG image, based on CGdescriptive data defining contents of a virtual three-dimensional spaceformed with CG; a three-dimensional operating unit configured to be usedfor inputting parameters of three axial directions perpendicular to oneanother for performing a set of predetermined operations on a virtualcamera and on an object in the virtual three-dimensional space; anassociating unit configured to provide an ID identifying the virtualcamera and to provide an ID identifying the object in the virtualthree-dimensional space, the associating unit being responsive to theselection of a provided ID to determine an object to be controlled suchthat the selection of the virtual camera ID selects the virtual cameraas the object to be controlled with the three-dimensional operating unitand the selection of the object ID selects the object in the virtualthree-dimensional space as the object to be controlled with thethree-dimensional operating unit; a coordinate mode selecting unitconfigured to select a target system or a source system as a coordinatemode indicating a coordinate system of an operation using thethree-dimensional operating unit; and a control unit configured to, inresponse to the input of the parameters of the three axial directionsthrough the three-dimensional operating unit, control the CG imagegenerating unit to cause an image of the virtual three-dimensional spaceto vary in response to one set of instructions when the virtual camerais selected as the object to be controlled with the three-dimensionaloperating unit and in response to another set of instructions when theobject in the virtual three-dimensional space is selected as the objectto be controlled with the three-dimensional operating unit, thevariation in the image of the virtual three-dimensional space beingcaused by changing the amount of control of the CG image generating unitin accordance with the input parameters, and wherein the one set ofinstructions allows some of the predetermined operations in the set tobe performed on the virtual camera and prohibits others of thepredetermined operations, and wherein the other set of instructionsallows at least the prohibited operations in the set to be performed onthe object in the virtual three-dimensional space.